Language Independent Feature Extractor

We propose a new customizable tool, Language Independent Feature Extractor (LIFE), which models the inherent patterns of any language and extracts relevant features of the language. There are two contributions of this work: (1) no labeled data is necessary to train LIFE (It works when a sufficient number of unlabeled documents are given), and (2) LIFE is designed to be applicable to any language. Although there are some studies that aim to design language independent feature extractors, we argue that most of them are not truly language independent. First, many works depend on some other resources or tools (e.g., WordNet) which themselves are inherently language-specific. In (Steinberger, Pouliquen, and Ignat 2006), many resources and features were employed, and a huge effort will be required to apply it to other languages. (Curran and Clark 2003) defined word-level features, alphabet-level features, and some features obtained from a gazetteer which again is language-specific. Many of these approaches are available only when these resources are constructed using the target languages. Second, most of these studies are applicable to alphabetbased languages (e.g., English), but not to non-alphabetbased languages (e.g., Korean, Chinese) because the characteristic difference between the two types of language is not considered. For example, Part-Of-Speech (POS) tags are usually allocated to each morpheme in Korean, while the POS tags in English are allocated to each word(token). There can be no blank space between morphemes or words in Korean and Chinese, while in English, a blank space separates two words. In order to design a truly language independent feature extractor, analysis of documents should not depend on such language-specific assumptions. There are several approaches (Chen et al. 2010; Jing et al. 2003) that do not depend on such language-specific assumption of word definition. Instead, they employ letterlevel features. (Jing et al. 2003) compares letter-level, wordlevel, and class-level features by their performances in NER task on Chinese language. The class-level features are defined to be the class tags of words such as numbers, Chinese names, foreign names, etc. In this work, the letter-level